Greptile Summary

This PR refactors Fireworks deployment health checking and cooldown tracking to be per-deployment rather than global, so that a degraded or scaling-up deployment for one model doesn't block routing to other models.

Key changes:

• fireworks-health.ts (classify): Switched from "worst-of" to "best-of" semantics — the waiting room now admits users if any deployment is healthy, and only blocks all users when every deployment is non-healthy.
• fireworks.ts: Replaced the single global deploymentScalingUpUntil timestamp with a Map<string, number> keyed by deployment path, so a DEPLOYMENT_SCALING_UP 503 on one deployment's cooldown doesn't bleed into other deployments.
• Tests: Added isolation tests for per-deployment cooldown and three new health classification scenarios (any-healthy, all-degraded, all-unhealthy).
• .gitignore: Added .gstack/.

Confidence Score: 5/5

Safe to merge — changes are well-scoped, backwards-compatible, and covered by new and updated tests.

The logic change in classify is straightforward and the best-of-any semantics align precisely with the deployment architecture (one deployment per model). The per-deployment cooldown Map is a clean, minimal refactor. tsc is clean, all 21 tests pass, and the three new health scenarios and two new cooldown-isolation tests provide solid coverage of the new behaviour. No security implications, no data-loss risk, and the fallback path in createFireworksRequestWithFallback already handles any individual deployment failure gracefully.

No files require special attention.

Important Files Changed

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Waiting Room: getFireworksHealth] --> B[probe: fetch Prometheus metrics]
    B --> C{deploymentIds empty?}
    C -- yes --> D[return 'healthy']
    C -- no --> E[classify samples, deploymentIds]

    E --> F{for each deploymentId\nclassifyOne}
    F --> G{any 'healthy'?}
    G -- yes --> D
    G -- no --> H{any 'degraded'?}
    H -- yes --> I[return 'degraded'\ndo NOT admit]
    H -- no --> J[return 'unhealthy'\ndo NOT admit]

    subgraph classifyOne
        K[KV blocks >= 0.98?] -- yes --> L[unhealthy]
        K -- no --> M[5xx rate >= 10%?]
        M -- yes --> L
        M -- no --> N[prefill p90 > 1000ms?]
        N -- yes --> O[degraded]
        N -- no --> P[KV blocks >= 0.80?]
        P -- yes --> O
        P -- no --> Q[healthy]
    end

    subgraph createFireworksRequestWithFallback
        R[isDeploymentCoolingDown deploymentId] -- cooling down --> S[standard Fireworks API]
        R -- not cooling down --> T[custom deployment request]
        T -- 503 DEPLOYMENT_SCALING_UP --> U[markDeploymentScalingUp deploymentId\ncooldown per-deployment Map]
        U --> S
        T -- other 5xx --> S
        T -- success --> V[return response]
    end

_{Reviews (1): Last reviewed commit: "Support multiple Fireworks deployments i..." | Re-trigger Greptile}